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(54) Titie: SELF-TUITION APPARATUS 
(57) Abstract 

Apparatus comprising input means (201). output 
means (202) and memory means (203) at least provided 
with an open-ended question memory (204) contain- 
ing open-ended questions and associated correct answers 
and a muldple-choice question memory (205) contain- 
ing multiple choice questions, which are linked to pre- 
determined open-ended questions, and associated mul- 
tiple-choice answers, and a processor (200). which is 
provided at least with first and second processing means, 
which first processing means are designed to: display 
at least one open-ended question via the output means 
(202); read a first answer input by the user, compare 
the first answer wid) the correct answer, the second pro- 
cessing means being activated automatically, if the first 
answer does not correspond to the correct answer, so as 
to carry out the following steps: displaying at least one 
multipIe-<hoice question with associated muUiple choice 
answers (202); reading a second answer input by the user 
(step 125); comparing the second answer with the correct 
answer. 



203- 



205> 



206. 



207- 



20U 



INPUT means! 



200 



[cm 



208' -TV 



202 



OUTPUT MEANSH 



209*^- 



210. 



OPEN-ENDED QUE8TI0N$ WITH 
CORRECT AND EXPECTED 
INCORRECT ANSWERS 



MULTIPLE-CHOICE QUESTIONS 
WITH ANSWERS 



ALGORITHM FOR ABSOLUTE 
COMPARISON WITH 
CORRECT ANSWERS 



ALGORITHM FOR ABSOLUTE 
COMPARISON WrTH EXPECTED 
INCORRECT ANSWERS 



ALGORITHM FOR RECOGNIZING 
ERRORS WHICH CAN BE IGNORED 



ALGORITHM FOR REL^TIVE 
COMPARISON WITH EXPECTED 
CORRECT ANSWERS 



ALGORITHM FOR RELATIVE 
COMPARISON WITH EXPECTED 
INCORRECT ANSWERS 



HINTS FOR USER 



211- 



12/20/05. EAST Version: 2.0.1.4 



FOR THE PURPOSES OF INFORMATION ONLY 



Codes used to identify States party to the PCT on the front pages of pamphlets publishing international applications under the PCX. 



AL 


Albania 


ES 


Spain 


LS 


Lesotho 


SI 


Sbvenia 


AM 


Anncnia 


FI 


Finland 


LT 


Lithuania 


SK 


Slovakia 


AT 


Austria 


FR 


France 


LU 


Luxembourg 


SN 


Senegal 


AU 


Australia 


GA 


Gabon 


LV 


Latvia 


SZ 


Swaziland 


AZ 


Azerbaijan 


GB 


United Kingdom 


MC 


Monaco 


TD 


Chad 


BA 


Bosnia and Heizegovina 


GB 


Georgia 


MD 


Republic of Moldova 


TG 


Togo 


BB 


Barbados 


GH 


Ghana 


MG 


Madagascar 


TJ 


Tajikistan 


BE 


Belgium 


GN 


Guinea 


MK 


TTie former Yugoslav 


TM 


Turkmenistan 


BF 


Burlctna Faso 


GR 


Greece 




Republic of Macedonia 


TR 


Turkey 


BG 


Bulgaria 


HU 


Hungary 


ML 


Mali 


TT 


TYinxlad and Tobago 


BJ 


Benin 


IE 


Ireland 


MN 


Mongolia 


UA 


Ukraine 


BR 


Brazil 


IL 


Israel 


MR 


Mauritania 


UG 


Uganda 


BY 


Belaius 


IS 


Iceland 


MW 


Malawi 


US 


United States of America 


CA 


Canada 


IT 


Italy 


MX 


Mexico 


UZ 


Uzbekistan 


CF 


Central African Republic 


JP 


Japan 


NE 


Niger 


VN 


Viet Nam 


CG 


Congo 


KE 


Kenya 


NL 


Netherlands 


YU 


Yugoslavia 


CH 


Switzerland 


KG 


Kyrgyzstan 


NO 


Norway 


ZW 


Zimbabwe 


CI 


Cdte d'lvoire 


KP 


Democratic People's 


NZ 


New Zealand 






CM 


Camertxjn 




Republic of Korea 


PL 


Poland 






CN 


China 


KR 


Republic of Korea 


PT 


Portugal 






CU 


Cuba 


KZ 


Kazakstan 


RO 


Romania 






cz 


Czech Republic 


LC 


Saint Lucia 


RU 


Russian Federation 






DE 


Germany 


U 


Liechtenstein 


SD 


Sudan 






DK 


Denmark 


LK 


Sri Lanka 


SE 


Sweden 






EE 


Estonia 


LR 


Liberia 


SG 


Singapore 







12/20/05, EAST Version: 2.0.1.4 



wo 98/32109 



PCT/NL98/00039 



1 

Self-tuition apparatus 

The present invention relates to a self -tuition apparatus. Vari- 
ous types of self -tuition apparatus are known in practice. They can be 
5 divided into two classes. The first class comprises apparatus which can 
be used to put open-ended questions to a student on a screen. The stu- 
dent answers the open-ended question with the aid of suitable input 
means, for example a keyboard. The apparatus comprises a memory, in 
which are stored not only the open-ended questions but also the associ- 

10 a ted correct answers. After the user has input his/her answer with the 
aid of the input means, the apparatus automatically compares the input 
answer with the correct answer belonging to the relevant open-ended 
question. If the two answers correspond, the apparatus generates a mess- 
age for the user in order to notify him/her of this fact, and the appar- 

15 atus then presents a following open-ended question. 

Another class of self-tuition apparatus relates to apparatus 
which can be used to present multiple-choice questions to a student. 
These multiple-choice questions are provided with various, for example 
four, multiple-choice answers. With the aid of the input means, the 

20 student selects the answer which he/she considers to be correct. In 
addition to the multiple-choice questions and the associated multiple- 
choice answers, the apparatus also stores an indication of which of the 
multiple-choice answers is the correct answer. The apparatus compares 
the answer input by the student with the correct answer and then noti- 

25 fies the student of whether he/she has answered the question correctly 
or incorrectly. Then, the apparatus automatically presents a following 
multiple-choice question with associated multiple-choice answers. 

The self -tuition apparatus of both classes have the drawback that 
they are primarily suitable for testing the knowledge of a student, and 

30 are not very suitable as an aid for stimulating the learning process. 

The object of the invention is to provide a self-tuition appar- 
atus which permits a new form of question-answer interaction of a 
strongly formative nature, i.e. in which questions put by the apparatus 
form part of the learning process and serve to a lesser extent as a test 

35 of the student's knowledge. 

To achieve this object, the present invention provides a self- 
tuition apparatus comprising input means for a user to input data, out- 
put means for outputting data to the user, memory means at least pro- 
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vided with an open-ended question memory containing open-ended questions 
and associated correct answers and a multiple-choice question memory 
containing multiple-choice questions, which are linked to predetermined 
open-ended questions, and associated multiple-choice answers, and a 
5 processor, which is connected to the input means, the output means and 
the memory means and is provided at least with first and second process- 
ing means, which first processing means are designed to: 
a) control a first user interface in the form of the display of at 
least one open-ended question via the output means; 
10 b) read a first answer input by the user via the input meems; 

c) compare the input first answer with the correct answer; 

and if the first answer read at step b) corresponds to the correct 
answer 

d) create a first message to the user and continue with step a) 
15 until all the open-ended questions have been answered, 

and the second processing means being activated automatically, if the 
first answer read at step b) does not correspond to the correct answer, 
so as to carry out the following steps: 

e) controlling a second user interface in the form of the display of 
20 at least one multiple-choice question with associated multiple- 
choice answers via the output means; 

f ) reading a second answer input by the user via the input means; 

g) comparing the second answer read at step f) with the correct 
answer; 

25 h) creating a second message to the user and continuing with step a) 
until all the open-ended questions have been answered. 
A self-tuition apparatus of this kind automatically decides the 
level at which questions are put to a student. If a student has learnt 
the study material well, he/she will answer most of the open-ended ques- 
30 tions correctly. If he/she does not know the answer to an open-ended 
question, the first processing means of the apparatus register this fact 
during the comparison step c) . The apparatus then automatically acti- 
vates the second processing means and presents the student with a mul- 
tiple-choice question with various multiple-choice answers. The relevant 
35 multiple-choice question corresponds to the open-ended question which 
the student answers incorrectly. 

The apparatus thus automatically controls the various user inter- 
faces and ensures that the various part-memories of the memory means are 



12/20/05, EAST Version: 2.0.1.4 



wo 98/32109 



PCT/NL98/00039 



3 

actuated. 

The new question-answer interaction which is presented here is 
intended specifically to be used in the context of computer-aided self- 
tuition. Use can advantageously be made here of modern technological 
5 developments. It is becoming every more possible to provide one computer 
per student, on which use can be made of multimedia presentation facil- 
ities, such as a soiand card, CD-ROM, and a very large memory capacity 
for information relating to individual students. The computers can be 
incorporated into a network. 

10 Preferably, authors of the questions, i.e. both the open-ended 

questions and the multiple-choice questions, operate in a context in 
which they themselves provide material for computer-aided self -tuition. 
To do this, the authors may use, for example, a windows word processor 
and specific graphical or linguistic tools. 

15 Preferred embodiments of the apparatus according to the invention 

are defined in the subclaims. 

The invention will be explained below with reference to a number 
of figures, which are intended only to illustrate the invention and not 
to limit the scope thereof. In the figures: 

20 Figure 1 shows a structure of a self -tuition apparatus according 

to the invention; 

Figure 2 shows a flow chart of one of the operating possibilities 
of the apparatus in accordance with Figure 1 , 

A self-tuition apparatus according to the invention comprises a 

25 central processor 200, which is denoted in Figure 1 by CPU. The central 
processor 200 is connected to suitable input means 201 and suitable out- 
put means 202. The input means 201 may relate to one or more desired 
options from the following possibilities: a keyboard, a mouse, a touch 
screen, as well as other known input means. 

30 The output means 202 relate to one or more options from output 

means which are known per se, such as a monitor, a display screen, audio 
means, etc. 

The self -tuition apparatus furthermore comprises a memory 203, 
which is likewise connected to the central processor 200, 
35 In its most basic form, the memory 203 comprises at least an 

open-ended question memory 204, in which open-ended questions and at 
least the associated correct answers are stored. Optionally, it is also 
possible to store expected incorrect answers here. This will be explain- 
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ed further below. The questions and answers may relate to written text, 
but also to pictures/ for example for children who are not yet able to 
read. 

The basic parts of the memory 203 also include a multiple-choice 
5 question memory 205 with associated multiple-choice answers / a memory 
206 which contains an algorithm for absolute comparison of the input 
answer with one or more correct answers, and a memory 211 which com- 
prises hints/messages for the user. 

In further, more detailed embodiments of the apparatus according 
10 to the invention, the memory 203 also comprises a memory 207, which con- 
tains an algorithm for absolute comparison of an answer input by a stu- 
dent with one or more expected incorrect answers, a memory 208 which 
contains an algorithm for recognizing errors which can be ignored, a 
memory 209 which contains an algorithm for relative comparison of the 
15 answer input by a student with the correct answer, and a memory 210 
which is provided with an algorithm for relative comparison of the ans- 
wer input by the student with expected incorrect answers. 

The following terms are of importance to the explanation of the 
principle of the question-answer interaction: 

20 - correct answer = correct response to the question asked; 

expected error = response by the student which corresponds 

to incorrect reasoning expected by the 
author of the questions; 

error of detail = response by the student which virtually 

corresponds to a correct answer or an 
expected incorrect answer; 

error = response by the student which does not cor- 

respond to one of the preceding possibil- 
ities. 

25 Figure 2 shows a flow chart to illustrate the operation of a pos- 

sible, very detailed functioning of the apparatus according to the in- 
vention. In step 101, the apparatus is started up, for example by the 
student. 

The apparatus responds by presenting an open-ended question from 
30 memory 204 via the output means 202, step 102. 

In step 103, the apparatus waits until the student has input an 
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answer . 

Once the student has input an answer, the apparatus compares the 
answer input with the correct answer. To do this, the apparatus employs 
the correct answers which are stored in the memory 204, together with 
5 the associated open-ended questions, and the algorithm stored in the 
memory 206 for absolute comparison with correct answers. Comparison 
algorithms of this kind are known to the person skilled in the art and 
do not need to be explained further here. If the answer input by the 
student is correct, a positive message is given to the student in step 
10 108. Then, the program returns to step 102 and a following open-ended 
question, if available, is presented to the student via the output means 
202. 

However, if step 104 finds that the student has not input a cor- 
rect answer with the aid of the input means 201, the program checks in 

15 step 105 whether the student has made an error of detail with respect to 
the correct answer. To do this, use can be made of known algorithms for 
graphical or linguistic recognition. The algorithm for graphical recog- 
nition is adapted to fit the technical features of the available input 
means and output means of which it is strongly dependent. The algorithm 

20 for linguistic recognition is fully integrated in the system and is 
therefore described in more detail here. The input of a linguistic 
answer can be made via a keyboard or via a microphone with a system for 
phonological conversion to text. The system is very suitable for the 
latest technological advanced form of input as the algorithm for lin- 

25 guistic recognition can filter features which are not significant. Such 
an algorithm for linguistic recognition filters the answer input by the 
student, checking for example for double spaces, spaces at the end of 
the answer, and the like. Then, the answer input by the student is com- 
pared in absolute terms with the correct answer with regard to the fol- 

30 lowing points: upper-case letters and lower-case letters, punctuation 
marks and diacritical marks. If the comparison shows that the student 
has only made an error of detail, the program proceeds to step 106. In 
step 106 it is established whether the student has made what is known as 
an error of detail which can be ignored. This may relate, for example, 

35 to the incorrect use of upper-case letters and lower-case letters. If 
the program establishes that the error of detail made can in fact be 
ignored, the program continues its route to step 108, which has already 
been explained above. 
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If the error of detail made is not an error of detail which can 
be ignored, the program proceeds to step 107, in which it provides the 
student, via the output means 202, with messages which can be used to 
help with correcting the error of detail. Then, the program returns to 
5 step 102. 

However, if it is found from the algorithm for graphical or lin- 
guistic recognition used that the student has not made an error of 
detail (step 105), the program proceeds with step 109. In step 109, the 
program makes use of an algorithm for absolute comparison of the answer 
10 input by the student with one or more expected incorrect answers. As 
stated earlier, an algorithm of this kind is stored in the memory 207. 
Incorrect answers of this kind are based, for example, on incorrect 
reasoning which is nevertheless obvious and is expected by the author of 
the questions. 

15 If the use of the algorithm for absolute comparison with the one 

or more expected incorrect answers shows that the student has in fact 
input an expected incorrect answer using the input means 201, the appar- 
atus notifies the student of this via the output means 202, via step 
110, in which the error of reasoning is pointed out to the student. 

20 The program then gives the student a hint on the correct answer, 

step 113. 

In step 114, the program waits until the student has input an 
answer, after having received a hint about it. 

After the student has input an answer, the program checks whether 
25 the input answer is correct. This is performed in step 115, which is 
identical to step 104. If step 115 shows that the student has input a 
correct answer, a positive message follows, step 116. After step 116, 
the program returns to step 102, if there are still open-ended questions 
left to be asked, 

30 However, if the program establishes in step 115 that a correct 

answer has still not been given, the program proceeds to step 124, in 
which a multiple-choice question with various multiple-choice answers is 
presented to the student instead of an open-ended question. This step 
124 will be explained below. As an alternative, step 117, which is ex- 

35 plained further below, can also follow here instead of step 124. 

If step 109 shows that the answer input by the student is not 
identical to an expected incorrect answer, the program checks in step 
111 whether the difference between the answer input by the student and 
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one or more expected incorrect answers is merely an error of detail. To 
do thiS/ use can be made of the algorithm for absolute comparison with 
one or more expected incorrect answers from memory 207. 

If it is established that the error is indeed an error of detail, 
5 the program checks in step 112 whether this is an error of detail which 
can be ignored. Step 112 is comparable to step 106^ except that the 
check here is for a difference from an expected incorrect answer, and 
not from the correct answer. If this is in fact an error of detail which 
can be ignored, the program proceeds with step 110, which has already 
10 been explained above. If it is not an error of detail which can be ig- 
nored, the program proceeds with step 117, which step is also carried 
out if it is established in step 111 that this is not an error of 
detail. 

It should also be noted that in steps 106 and 112 for establish- 

15 ing an error of detail which can be ignored use is made of an algorithm 
for recognizing errors which can be ignored, which algorithm is stored 
here in a separate memory 208. 

In step 117, the program carries out a relative comparison 
between the answer input by the student and the correct answer. To do 

20 this, use is made of an algorithm for relative comparison with correct 
answers, which algorithm is stored in memory 209. Algorithms of this 
kind are known in practice. Linguistic error recognition algorithms of 
this kind work at letter level. They have been used since approximately 
1980, for example in educational programs for typing and word processing 

25 of the types QUERTY, Vingervlug, WP-Trainer, AZtekst and TypPlus. Error 
recognition algorithms of this kind establish very accurately whether 
the student has input too many letters via the input means 201, whether 
there are letters missing, and other types of discrepancies. The algo- 
rithms provide a pattern which can be used to present the spelling cor- 

30 rection and to assess the relationship between the answer input by the 
student and the correct answer. This assessment may take the form, for 
example, of a number. 

It is then established in step 118 whether this number, which is 
a measure of the relative error observed, is less than or equal to a 

35 predetermined threshold value. 

If so, the program continues the procedure by giving a hint as to 
the correct answer via the output means 202, step 119. In step 120, the 
program then waits until the student has input an answer on the basis of 
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the hint given. Once this has been done, the program proceeds with step 
115, which has already been explained above. 

If it is fotind in step 118 that the relative error is greater 
than the predetermined threshold, the program continues with step 121, 
5 in which the program carries out a relative comparison with one or more 
expected incorrect answers. To do this, use is made o£ the algorithm for 
relative comparison with incorrect expected answers which is stored in 
memory 210. This error recognition algorithm may be of similar type to 
the error recognition algorithm which is stored in memory 209 and acts 
10 at letter level, in most cases, the algorithms from the memories 209 and 
210 will be identical and only have to be stored once. 

In step 122, the program checks whether the calculated relative 
error generated by step 121 is less than or equal to a predetermined 
threshold value. If so, the program continues with step 123, in which 
15 the student is given a hint about the correct answer. Then, the program 
proceeds to step 120, which has already been explained above. 

If the program establishes that the relative error in step 122 is 
greater than the predetermined threshold value, the program continues 
with step 124, in which it presents the student with a multiple-choice 
20 question with various multiple-choice answers, via the output means 202. 
A presentation of this kind may take the usual form, in which, for 
example, one multiple-choice question with associated multiple-choice 
answers is shown to the user on a monitor. 

In step 125, the program then waits until the user has input an 
25 answer with the aid of the input means 201 . As is known, the answer here 
consists in selecting one of the multiple-choice answers presented. 

In step 126, the program checks automatically whether the answer 
input by the student is correct. To do this, use can be madei for 
example, of an algorithm for absolute comparison with correct answers, 
30 which is stored in the memory 206. 

If the answer is correct, a positive message follows via the out- 
put means 202, step 127. However, if the answer input by the student is 
incorrect, there follows a negative message via the output means 202, in 
step 128, After step 127, the program returns to step 102. After step 
35 128, the program preferably returns to step 125 for as many times as 
necessary for the student to select the correct answer. 

It will be clear that the flow diagram shown in Figure 2 is a 
detailed embodiment of the functioning of the apparatus according to the 
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invention. In its most simple form, the steps of the program relating to 
checking for errors of detail, steps 105-107, the steps relating to 
checking the answer input by the student with respect to one or more ex- 
pected incorrect answers, steps 109-116, and the steps relating to car- 
5 rying out a relative comparison of the answer input by the st\ident with 
the correct answer or with one or more expected incorrect answers, steps 
117-123, are omitted. Thus in its most simple form, step 124, in which a 
multiple-choice question with various multiple-choice answers is pres- 
ented to the student, directly follows step 104, if it is established 

10 that the answer input by the student is incorrect. 

Checking for errors of detail, comparing with one or more ex- 
pected incorrect answers and carrying out relative comparisons with the 
correct answer or with one or more expected incorrect answers relates to 
options which refine the apparatus. 

15 With regard to the self-tuition apparatus presented above, the 

author of the open-ended questions and the multiple-choice questions, 
the student and any instructor have different roles. 

The author thinks up an open-ended question and an associated 
multiple-choice question, as well as one or more correct answers, ex- 

20 pected incorrect answers, for which a sensible explanation can be given, 
and multiple-choice answers. 

The student is initially presented with these questions as open- 
ended question via the output means 202 . It is known that open-ended 
questions test the knowledge of students at the most difficult level. 

25 The apparatus helps the student in all kinds of different ways to give 
the correct answer, for example using (linguistic) spelling tips, indi- 
cations of punctuation marks, automatic filling-in of non-core words, 
(graphically) showing contours, masking background, etc. Only if the 
student does not succeed in giving the correct answer at the level of 

30 the open-ended question (step 104) is the question presented as a mul- 
tiple-choice question with various multiple-choice answers. With the 
multiple-choice question, the student has the option, for example, of 
selecting an answer as many times as the number of multiple-choice ans- 
wers presented, 

35 It should be noted that whatever route the student follows, he/- 

she will always ultimately be aware of the correct answer. 

The apparatus of the present invention is preferably provided 
with means for recording the input answers and the route which the stu- 
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dent took for each question. It is possible in a simple manner to store 
in the memory 203 at what level and via which route the student gave the 
correct answer. The apparatus is preferably provided with means for com- 
piling graphs or the like on the basis of the above factors, which 
5 graphs can be called up by the instructor and can be shown on the output 
means 202. The routes followed by the student in responding to the ques- 
tions can, for example, be "played back" again in the presence of the 
instructor, so that the instructor is helped in his/her discussion of 
the teaching material with the student. 

10 The apparatus according to the invention has various advantages. 

For example, the author only has to take into account the answers 
which he/she finds useful in furthering the teaching process. He/she 
only has to compile correct answers or answers which are incorrect for 
an obvious reason. Moreover, in the multiple-choice questions the author 

15 does not have to invent extra diversionary answers to avoid the guessing 
effect. This is because the student is always asked at the level of the 
open-ended questions to input the correct answer. 

Furthermore, each route travelled by the student for each ques- 
tion leads to a meaningful answer. The questions asked by the apparatus 

20 form part of the teaching material and can be assessed. There is less 
need for tests. 

The interaction between the student and the apparatus has both a 
high maximum result, namely a correct answer immediately after an open- 
ended question has been asked, as well as a positive minimum result, 

25 namely finding the correct answer via the multiple-choice questions. The 
student learns even from this. By recording the routes followed by the 
student, an instructor can establish where the student has problems with 
the teaching material. By calling up the relevant routes, the student 
can offer help in solving these problems. 

30 The apparatus discussed thus forms a teaching aid, since it is 

able to switch automatically between various layers of questions and 
associated answers in interaction with a student. 
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Claims 



1. Self-tuition apparatus comprising input means (201) for a user to 
input data, output means (202) for outputting data to the user, memory 

5 means (203> at least provided with an open-ended question memory (204) 
containing open-ended questions and associated correct answers and a 
multiple-choice question memory (205) containing multiple-choice ques- 
tions, which are linked to predetermined open-ended questions, and asso- 
ciated multiple-choice answers, and a processor (200), which is connect- 
10 ed to the input means (201), the output means (202) and the memory means 
(203) and is provided at least with first and second processing means, 
which first processing means are designed to: 

a) control a first user interface in the form of the display of at 
least one open-ended question via the output means (202) (step 

15 102)V 

b) read a first answer input by the user via the input means (201) 
(step 103); 

c) compare the input first answer with the correct answer (step 
104); 

20 and if the first answer read at step b) corresponds to the correct 
answer 

d) create a first message to the user (step 108) and continue with 
step a) until all the open-ended questions have been answered, 

and the second processing means being activated automatically, if the 
25 first answer read at step b) does not correspond to the correct answer, 
so as to carry out the following steps: 

e) controlling a second user interface in the form of the display of 
at least one multiple-choice question with associated multiple- 
choice answers via the output means (202) (step 124); 

30 f) reading a second answer input by the user via the input means 
(201) (step 125); 

g) comparing the second answer read at step f) with the correct 
answer ( s tep 126); 

h) creating a second message to the user (step 127/128) and continu- 
35 ing with step a) until all the open-ended questions have been 

answered . 

2. Apparatus according to Claim 1, characterized in that firstly, 
before steps e) to h) are carried out, third processing means are auto- 
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raatically activated so as to carry out the following steps: 
i) establishing (step 105) whether the difference between the first 
answer and the correct answer is a first error of detail which 
can be ignored, and if so 
5 i1) creating a message (step 107/108) for the user and continu- 

ing with step a) until all the open-ended questions have 
been answered, 
and if not: 

i2) automatically activating the second processing means so as 
10 to carry out steps e) to h) . 

3. Apparatus according to Claim 1 or 2, characterized in that the 
memory means (203) are also provided with at least one expected incor- 
rect answer belonging to the open-ended questions, and in that firstly, 
before steps e) to h) are carried out, fourth processing means are acti- 

15 vated so as to carry out the following steps: 

j) comparing the input first answer with the at least one expected 

incorrect answer (step 109); 
and if the input first answer corresponds to the at least one expected 
incorrect answer 

20 j1) giving a hint to the user via the output means (step 113); 

j2) reading a third answer input by the user via the input means 

(201) (step 114); 
j3) comparing the input third answer with the correct answer 

(step 115); 

25 j4) if the input third answer is correct, giving a message to 

the user (step 116) and continuing with step a) until all 
the open-ended questions have been answered; 
and if both the first answer does not correspond to the at least one 
expected incorrect answer and the third answer does not correspond to 
30 the correct answer 

j5) automatically activating the second processing means so as 
to carry out steps e) to h) , 

4. Apparatus according to Claim 3, characterized in that firstly, 
before step j5) is carried out, fifth processing means are automatically 

35 activated so as to carry out the following steps: 

k) establishing (step 111) whether the difference between the first 
answer and the at least one expected incorrect answer is a second 
error of detail which can be ignored, and if so 
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k1 ) carrying out steps j1) to j3) and, if the third answer then 
does not correspond to the correct answer, carrying out step 
j5); 

and if not 

5 k2) automatically activating the second processing means so as 

to carry out steps e) to h) . 

5. Apparatus according to one of the preceding claims, characterized 
in that firstly, before steps e) to h) are carried out, sixth processing 
means are automatically activated so as to carry out the following 

1 0 steps : 

1) comparing in relative terms the input first answer with the cor- 
rect answer and calculating a first relative error (step 117); 
m) comparing the first relative error with a predetermined first 
threshold value (step 118); 
15 and if the first relative error is not greater than the threshold value 
ml) giving a hint to the user via the output means (step 119); 
m2) reading a fourth answer input by the user via the input 

means (201) (step 120); 
m3) comparing the input fourth answer with the correct answer 
20 (step 115); 

m4) if the input fourth answer is correct, giving a message to 
the user (step 116) and continuing with step a) until all 
the open-ended questions have been answered; 
and if either the first relative error is greater than the first thresh- 
25 old value or the fourth answer does not correspond to the correct answer 
m5) automatically activating the second processing means so as 
to carry out steps e) to h) . 

6. Apparatus according to Claim 6, characterized in that the memory 
means (203) are also provided with at least one expected incorrect 

30 answer belonging to the open-ended questions, and in that, if the first 
relative error is greater than the first threshold value, firstly, 
before step mS) is carried out, seventh processing means are activated 
so as to carry out the following steps: 

n) comparing in relative terms the input first answer with the at 
35 least one expected incorrect answer and calculating a second 

relative error (step 121); 
o) comparing the second relative error with a predetermined second 

threshold value (step 122); 
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and if the second relative error is not greater than the second thresh- 
old value 

01 ) giving a hint to the user via the output means (step 123); 

02) reading a fourth answer input by the user via the input 
•means (201) (step 120); 

03) comparing the input fourth answer with the correct answer 
(step 115); 

04) if the input foxirth answer is correct, giving a message to 
the user (step 116) and continuing with step a) until all 
the open-ended questions have been answered; 

and if either the second relative error is greater than the second thre- 
shold value or the fourth answer does not correspond to the correct 
answer 

05) automatically activating the second processing means so as 
* to carry out steps e) to h) . 

7. Apparatus according to one of the preceding claims, characterized 
in that all the answers input by the user via the input means are auto- 
matically stored in the memory means (203). 

***** 



12/20/05, EAST Version: 2.0.1.4 



wo 98/32109 



PCT/N198/00039 



1/2 



fiq-l 



203 



201. 



INPUT MEANS 



200- 



20^ 



205- 



205. 



207. 



208 



CPU 



202 



OUTPUT MEANS 



209 



210. 



OPEN-ENDED QUESTIONS WITH 
CORRECT AND EXPECTED 
INCORRECT ANSWERS 



MULTIPLE-CHOICE QUESTIONS 
WITH ANSWERS 



ALGORITHM FOR ABSOLUTE 
COMPARISON WITH 
CORRECT ANSWERS 



ALGORITHM FOR ABSOLUTE 
COMPARISON WITH EXPECTED 
INCORRECT ANSWERS 



ALGORITHM FOR RECOGNIZING 
ERRORS WHICH CAN BE IGNORED 



ALGORITHM FOR RELATIVE 
COMPARISON WITH EXPECTED 
CORRECT ANSWERS 



ALGORITHM FOR RELATIVE 
COMPARISON WITH EXPECTED 
INCORRECT ANSWERS 



HINTS FOR USER 



211 



12/20/05, EAST Version: 2.0.1.4 



wo 98/32109 



PCT/NL98/00039 




12/20/0*5, EAST Version: 2.0.1.4 



INTERNATIONAL SEARCH REPORT 



tjTtet onal Application No 

PCT/NL 98/00039 



A. CLASSIFICATION OP.SUBJECT MATTER , 

IPC 6 G09B7/04 G09B7/05 



According to intemattonal Patent ClaasificationnPC) or to both national ctaa&iflcation and IPC 



B. FIELDS SEARCHED 



Minimum documentation searched (classification system foSowed t}y ctasstfication synAols) 

IPC 6 G09B 



Documentation searched other than minimum documentation to the extent that such documema are Included in the Melds searched 



Electronic data base cor^ultad during the inteiTiatiortal search (name of data base and, where practical, search terms used) 



C. DOCUMENTS CONSIDERED TO BE RELEVANT 



Category * Citation of document, with Indication, where appropriate, of the relevant passages 



Relevant to claim No. 



us 5 616 033 A (KERWIN PATRICK A) 1 April 
1997 

see column 2, line 42 - column 6, line 27; 
claims 1-20; figures 1-2B 

US 5 618 182 A (THOMAS C DOUGLASS) 8 April 
1997 

see column 3, line 43 - column 10, line 
17; claims 1-3,7,816,17.21 

US 5 590 057 A (FLETCHER ROBERT L ET AL) 
31 December 1996 

see column 3, line 24 - column 5, line 64; 
claims 1-4 



-/-- 



1.7 



1.5 



Further documents are listed in the continuation of box C. 



Patent family members are listed in annex 



" Special categories of cited documents : 

"A" document defining the general state of the art which is not 

considered to be of particular relevance 
"E* earlier document but pub&shed on or after the international 

ftDng date 

"L" document which may throw doubts on priority claim(s) or 
which is cited to establish the publication date of another 
citation or other special reason (as specified) 

'C document referring to an oral disclosure, use, exhibition or 
other means 

"P" document pubfished prior to the International filing date but 
later than the priority date claimed 



T" later document published after the International filing date 
or priority date and not in confiid with the application but 
cited to understand the principle or theory underiying the 
invention 

"X" document of particular relevance; the claimed invention 
cannot be considered novel or cannot be considered to 
involve an inventive step when the document Is taken alone 

"Y" document of particular relevarvce; the claimed Invention 

cannot be considered to Involve an inventive step when the 
document is combined with one or more other such docu- 
ments, such combination being obvious to a person sxmed 
in the art 

*&" document member of the same patent family 



Date ot the actual completion of the international search 



20 March 1998 



Date of maillr^ of the international search report 



26/03/1998 



Name and mailing addrasa of the ISA 

European Patent Office. P.B. 5818 Patentlaan 2 
NL - 2280 HV Rljswijk 
Tel. (+31-70) 340-2040. Tx. 31 651 epo nl, 
Fax: (+31-70) 340-3016 



Authorized officer 



Gorun, M 



Form PCT/tSA/210 (second Sheet) (J»iyl9?2.* — *^ - ^ 

12/20/05, EAST Version: 2.0.1.4 



INTERNATIONAL SEARCH REPORT 



tntei 3nBl Application No 

PCT/NL 98/00039 



C.{Continu«tlon) DOCUMENTS CONSIDERED TO BE RELEVANT 


Categcry ^ 


Citation of document with Indication. where appropriate, ot the relevant passages 


Relevant to claim No. 



us 5 180 309 A (EGNOR MICHAEL E) 19 
January 1993 

see column 1, line 57 - column 3, line 58 



US 5 577 919 A (COLLINS DEBORAH L ET AL) 
26 November 1996 

see column 2, line 14 - column 10, line 
67; claims 1,4,7 

US 5 421 730 A (LASKER III HARRY M ET AL) 
6 June 1995 

see the whole document 

US 4 958 284 A (BISHOP LAWRENCE C ET AL) 

18 September 1990 

see the whole document 

MO 95 08810 A (EMPOWER INFORMATION 
TECHNOLOGY ;WRIGHT GREGORY ALLAN (AU)) 30 
March '1995 



1 

2-5 
1 



1 



Farm PCT/ISAAIO (continuation ot second shoei) (July 1092) 

12/20/05, EAST Version: 2.0.1.-4,aae ? of 7 



INTERNATIONAL SEARCH REPORT 

infonnatton on patent (amity member* 



InU ional Application No 

PCT/NL 98/00039 



Patsnt document 
cited in search report 


ruoucaQon 
date 


raVSill loiniiy 

memt>er(s) 


r^UUllCaUUf 1 

date 


US 5616033 A 


01-04-97 


NONE 




US 5618182 A 


08-04-97 


NONE 




US 5590057 A 


31-12-96 


NONE 




US 5180309- A 


19-01-93 


NONE 




US 5577919 A 


26-11-96 


US 5437553 A 
WO 9637869 A 
US 5545044 A 
AU 2650295 A 


01-08-95 
28-11-96 
13-08-96 
11-12-96 


US 5421730 A 


06-06-95 


NONE 




US 4958284 A 


18-09-90 


NONE 




WO 9508810 A 


30-03-95 


AU 7733594 A 


10-04-95 



Form PCT/ISA«10 (pitomfamtty innox) (July 1M2) 

12/20/05, EAST Version: 2. 0. 1.4 



